<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->
<!-- not a package-info.java, because we already defined this package in spatial/ -->
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body>
 Fast, general-purpose grammar-based tokenizers.
 <ul>
     <li>{@link org.apache.lucene.analysis.standard.ClassicTokenizer ClassicTokenizer}:
         this class was formerly (prior to Lucene 3.1) named 
         <code>StandardTokenizer</code>.  (Its tokenization rules are not
         based on the Unicode Text Segmentation algorithm.)
         {@link org.apache.lucene.analysis.standard.ClassicAnalyzer ClassicAnalyzer} includes
         {@link org.apache.lucene.analysis.standard.ClassicTokenizer ClassicTokenizer},
         {@link org.apache.lucene.analysis.standard.StandardFilter StandardFilter}, 
         {@link org.apache.lucene.analysis.LowerCaseFilter LowerCaseFilter}
         and {@link org.apache.lucene.analysis.StopFilter StopFilter}.
     </li>
     <li>{@link org.apache.lucene.analysis.standard.UAX29URLEmailTokenizer UAX29URLEmailTokenizer}:
         implements the Word Break rules from the Unicode Text Segmentation
         algorithm, as specified in 
         <a href="http://unicode.org/reports/tr29/">Unicode Standard Annex #29</a>, except
         URLs and email addresses are also tokenized according to the relevant RFCs.
         <br>
         {@link org.apache.lucene.analysis.standard.UAX29URLEmailAnalyzer UAX29URLEmailAnalyzer} includes
         {@link org.apache.lucene.analysis.standard.UAX29URLEmailTokenizer UAX29URLEmailTokenizer},
         {@link org.apache.lucene.analysis.standard.StandardFilter StandardFilter},
         {@link org.apache.lucene.analysis.LowerCaseFilter LowerCaseFilter}
         and {@link org.apache.lucene.analysis.StopFilter StopFilter}.
     </li>
 </ul>
 <p>
 This Java package additionally contains {@code StandardAnalyzer}, {@code StandardTokenizer},
 and {@code StandardFilter}, which are not visible here, because they moved to Lucene Core.
 The factories for those components (e.g., used in Solr) are still part of this module.
</body>
</html>
